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ABSTRACT 

A program for trained vocational education curriculum 
. ^specialists (VECS), consisting of 16 modules, was written, revised, 
and field tested aJt 15 sites nationwide. The instructional materials 
were written to deliver the highest rated competencies based on, a 
field survey of vocational educators and review by a national 
advisory panel of vocational education experts. VECS modules were 
designed to create or upgrade an individual's vocational education 
curriculum development and management skills. Additional materials 
developed were a guide for instructors and administrators and audio 

modified .quasi-experimental, pretest/posttest , treatment 
group/control group design was used. Participants were undergraduates 
in vocational education teacher preparation, practicing vocational 
educators, and persons with occupational skills who wished to. teach 
their specialty at" 12 colleges/universities^ and two state departments 
of education. Field test evaluation forms were developed to measure 
cognitive and affective outcomes and to collect biographical 
information. Results of .the national field test 
modules increased knowledge of topics necessary 
'performance of skills central to the VECS rolfe. 
increase peoples' 'Confidence in their ability to 



demonstrated that the 
to the successful 
They % also tended to 
perform these 



skills. 
(YLB) 
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A program for training vocational education curriculum specialists 
(VEQS), consisting of 16 modules, has £een written, revised, and field 
tested at ,15 sites across the country. Curriculum specialists are trained 
to develop and manage ciirricula so that vocational programs meet the needs 
of individual students, respond to current vocational education legisla- 
tion, .and provide skilled workers for the labor market. The VECS training 
modules cover topics in the history and philosophy of vocational education, 
and in curriculum envelopment, management, and evaluation. Each module 
includes goals and objectives, text, learning activities, lists of 
resources, and self-check items. The modules can be used i* classes or 
individual study arrangements at 'the preservice or inservice level by stu- 
dents with varying amounts of experience in vocational education. 



Module Development 

T^he VECS modules were written to meet an acknowledged national need. 
Emerging social priorities and changing employment and training require- 
ments produced by technological advances have resulted in a demand for the 
improvement of vocational 'training programs. Federal legislation since 
1968 has, mandated expanded vocational education curricula for a wide 
variety .of students. At a federally funded Conference on Curriculum Devel- 
opment in Vocational and Technical Education held early in 1969, the train- 
ing of specialists in curriculum theory, design, development, management, 
and evaluation to achieve curriculum reform was identified as an essential 
national priority. 

* * The United States Office of Education (USOE) held internal planning 
conferences in 1972-73, resulting in a decision to fund the design of voca- 
tional education curriculum specialist training programs. In 1974, the 
Bureau *of Occupational and Adult Education (BOAE) awarded contracts to the\ 
American Institutes for Research (AIR) and to Washington State University 
(WSU) to produce, test, and revise such materials. In 1978, BOAE again 
contracted with AIR, this time to conduct a national field test of the VECS 
materials. The overall goal of this project was to integrate the two sets 
of VECS "materials produced by AIR and WSU, systematically field test them, 

• and -encourage their use in the field. 

In the first two contracts, primary emphasis was placed on the devel- 
opment of the VECS modules. In (designing the firsf sets of materials, 
careful analyses of 'the competencies required of a curriculum specialist 
were conducted based on a field survey of vocational educators and Ja review 

* by a x national advisory panel of vocational education experts. The instruc- 
tional materials were written to deliver the hi-ghest -rated competencies. 
Prototype units were critiqued by consultants, instructors, and 4 students, 
and then tested. Based on these results, content organization and format- 
ting were revised, and formal developments and production initiated. The 
resulting materials were o subsequently pilot tested at five universities by 
an. independent third-party evWluator. On the basis of the pilot test 
results, the materials then underwent a second revision. 



AIR assumed that the competency base for the first sets of VECS 
materials was* sound. Thus, (efforts in the national field test concentrated 
on integrating and updating the first AIR and WSU materials, and on assur- 
ing that the resulting materials were appropriate for use in a wide variety 
of settings with varied target populations. Efforts in the. third revision 
cycle pripr to the national field test improved strategies for presenting 
information*, added learning activities and support materials, eliminated 
"overlap ^nd redundancy, and focused on current national priorities regard- 
ing sex equity and training for handicapped and older students. Suggestions 
for * improving the modules were collect/ed during the national field test 
from instructors, students, and. consultants in vocational education profes- 
sional development. Based on this inftut, the moduJLes were extensively 
edited and reordered, and a user's guide ( M Using the VECS Modules, 11 1981) 
was prepared to summarize implementatioiNstrategies. 




Purpose of the Modules 



When. the VECS projects were first conceived, their purpose was to pre- 
pare training materials for a, new kind of specialist, the vocational educa- 
tion curriculum specialist, whose exclusive duty would be to take charge of 
all vocational. curriculum development and management functions within an 
educational agency or institution. The vocational education curriculum 
specialist would work with teachers and advisory committees, pulling > 
.together suggestions and resources, to develop new curriculum ideas and 
assist in their implementation. 

C ■ 

As the VECS modules evolved over the past six years, it became clear 
,thaf full-time specialists with responsibility only for vocational curricu- 
lum are employed at a limited number of instructional materials centers 
around the country. Further, it is evident that vocational* educators at 
many levels in tt\e educational hierarchy need curriculum skills. 

When funding is available and State or local organizational structures 
permit, 1 full-time vocational education curriculum specialises are employed. 
More typticaliy, many, different individuals share the curriculum special- 
ist's responsibilities. They Inay be researchers or they may hold various 
job titles "such as: .dean of occupational education; local director of 
vocational education; department or division director for vocational educa- 
tion; supervisor of occupational' instruction; program supervisor for a 
vocational education discipline; or principal or assistant principal of a 
vocational-technical school ox area vocational center. Even vocational 
instructors need many of th^ skills of curriculum specialists in their 
classrooms. 

• Thus, when used in the context of vocational education curricula, 
the original definition of the term "specialist" has been broadened and 
expanded. In the VECS modules, the words "curriculum specialist" or "voca- 
tional education curriculum specialist" are .used as an easy way to refer to 
any vocational educator with responsibility for major activities related to 
curriculum development and management at the local, district, ot state 
level, vhether or not he or she is a "specialist" in the traditional* sense. 
All vocational educators should participate, to some degree, in making 
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' * ' TABLE I 

• ■* 

. ' v , * \ 

VECSTrtodiile Titles 



Introdflkory 

nodule 


I. 


— 7 " 

Vocational Educators and Curriculum Management 




2. 


The Scope of Vocational Education 


Foundations *^ 
Series 

*> 


3. 
4 . 
5. 


* Organization of Vocational Education 
Legislative Mandates for Vocational Education 
Priorities in Vocational Education 


% 

. ! 


e. 

o . 


Vocational Education for Students with Special Needs 




7. 


Vocational. Needs Assessment antf Curriculum Development 


Curriculum^ 


8. 


Conducting Task Analyses and Developing Instructional Objectives 


Development 
Series 


9. 


Selecting Instructional Strategies and Assessing Student Achievement 


* 


10. 


Relating Learning Differences and Instructional Methods 




11 . 


Selecting and Preparing Instructional Materials | >c 
^ ^ S- 




12. 


Evaluating Vocational Education Curricula 


Admin 1st rnt Ive 
Series 


13. 


Conducting Pollow-fl^ Studies and Communicating Evaluation Results 
Managing Vocational Education Programs 


* * • 


15. 


Preparing for Curriculum Change 




16. 


Staff Development 



Modules average aboii^75 pages in length and each requires abou£ 30 to 
urs to complete if ^11 readings ^id activities 'are done thoroughly. 
Modules follow a standard format, whj^h includes a detailed listvpf beha- 



vioral goals and object xves-r=aitd" provip^s text, learning activities, and 
^s.elf-assessment i tems re la^fred tb each goal. The modules usually contain 

from thr&e to five goals each. Learning activities include reading in 

published resources, group projects, and discussion questions. Interviews 
-with %votrartionai* -educator-s -and actual practice in curriculum development 

skills are frequently called for. Self-assessment items provide students 

with feedback on thew: „ learning of importf&ht topics. 

*t A user's guide h^s been written to accompany the modules. It describes 
the various use patterns that the. modules /can accommodate. The whole set 
'of modules can be instituted a# the foundation of a formal curriculum**Spe- 
ciglist training program,, of* selected groups of modules can serve as 'the 
focus of specific courses/ In addition, components of modules, individual 
modules,, or series of modules can be integrated into existing courses, or 
programs. 1 * Because the^modules are organized around specific goals and 
include alternative learning activities, their use is appropriate in tradi- 
tional aqd nontrdditional classrooms, inservice education programs, and 
independent sfcudy. 
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TABLE I 
VEC^itodule Title 



Vocational Educators and Curriculum Management 



IntrodWlory m I . 

Module 4 



Foundations 
Series 



,2. The Scope t of Vocational Education 

3. ♦ Organization of Vocational Education 

4. Legislative Mandates for Vocational Education 

5. Priorities in Vocational Education 

6. Vocational Education for Students with Special Needs 



Curriculum^ 
Development 
Series 



Adminlstrat Ive 
Series 



7. 
8. 
9. 
10. 
11 . 



Vocational* Needs Assessment antf Curriculum Development 

Conducting Task Analyses and Developing Instructional Objectives 

Selecting Instructional Strategies and Assessing Student Achievement 

Relating Learning Differences and Instructional Method^s 

Selecting and Preparing Instructional Materials k >t ^» 



12. Evaluating Vocational Education Curricula 

13. Conducting Follow-fl^ Studies and Communicating Evaluation Results 

14. Hnnagttig Vbcational Education Programs 

15. Preparing for Curriculum Change 

16. Staff Development 



Modules average abotf£^75 pages in length and each requires abou£ 30 to 
50 hours to complete if ^11 readings ^id activities 'are done thoroughly. 
Modules follow a standard format, wh^h includes a detailed list^pf beha- 
vioral goals and object iv e a, and " provip^s text, learning activities, and 

^s.elf-rassessment items fela'fred tb each goal. The modules usually contain 
from thr&e to .five goals each. Learning activities include reading in 
published resouj^es, group projects, and discussion questions. Interviews 

-with%vocfftion^i* -educators *-atkl actual practice in curriculum development 
skills are frequently called for. Self-assessment items provide students 
with feedback on their , learning of imporCtht topics. 

A user f s guide h^s been written to accompany the modules. It describes 
the various use patterns that the, modules /can accommodate. The whole set 
'of modules can be instituted a# the foundation of a formal curriculum^pe- 
cialist training program, of selected groups of modules can serve as the 
focus oi specific courses/ In addition, components of modules, individual 
modules,, or series of modules can be integrated into existing courses, or 
programs. 1 * Because the^modules are organized around specific goals and 
include alternative learning activities, their use is appropriate in tradi- 
tional aqd riontrdditional classrooms, inservice education programs, and 
independent sfcudy. r 



-4- 



V 



Field Test Design * 

* . 

When the field tfest ,was first ^conceptualized , AIR ^taff planned to use 
a pre test/posttest , treatment group/control^ group design to test the effec- 
tiveness of the modules. As planning progressed-, however, the importance 
of conducting the ^field test under the types of conditions expected to 
represent actual module use became increasingly apparent. AIR staff chose 
to conduct the field test under natural conditions; this decision precluded 
exercising .the type of control over schedule, selection of subjects, tre.atT 
ment, and testing that is necessary to carry />ut a scientifically rigorous 
research design. -* , 

Several factory operated against conducting the field test according 
to textbook procedures. For example, it was necessary to select sites 
based on the appropriateness of the setting and the student population, and 
on their willingness to cooperate in the field test. Random selection 
procedures could not be employed. A local coordinator at each aJLte identi- 
fied instructors and students to serve in the field test. For the most 
part, intact classes wer'e selected. Again, random selecTtion proved imprac- 
tical. Although instructors received guidelines on >ow to use modules in 
their classes during an orientation session, instructors adapted the use of 
the modules tb meet their own needs. Implementation of moHules across 
sites could not be standardized; students did not receive the same' treat- 
ment.' Because of the "naturalistic 11 character of the field test,, AIR staff 
could exercise little control over the schedule of module use, the method 
of module implementation, or the selection and testing of participating 
students. 

In planning the field tes't, AIR staff sought, to gather control group 
data from students who would be tested but who would not' study any modules. 
Ideally, the control group would be tested a,t the ? same times the treatment 
group was pre- and posttested. However, because control group students . 
would not study the modules, it was likely that their motivation for con- 
scientiously completing tests that were only ^partially related to thei<r > 
course work would be low. TShus, we could not expect to obtain-^dequate 
pretest/'posttest data from a control group. In order to increase the prob- 
ability of obtaining some useable data, AIR staff decided to test cdntrdL 
students only once. At^some sites this testing occur^fd when* the treatment 
group was pretested; at othe£ sites the control group testing occured when 
treatment students were posttested. We chose to pretest some controls and 
postjt^est others so that some of the benefits of a pretest/posttest , treat- 
ment group/ccmtro'l group design might be realized. Unfortunately, because 
of problems with the comparability of treatment and octroi groups, it was 
appropriate to comiucfc-only a limited number of analyses using Control 
group data. While the results of these analyses lend some sup.port to tFie 
conclusions drawn about the effectiveness of the VECS modules, they will 
not be discussed here. The treatment group/control group analyses that , 
were run'represent an incomplete set, and this type of analysis is not of 
primary importance in drawing conclusi(Jns about module effectiveness^ The 
datl reported here are those that would result from* a one group, pretest^ 
posctest design. / 

/ "• -. • 



The modules were tested in two phases at 15 sites. The^ desigfi featured 
multiple replications with .different types of students using various pat- 
terns of module implementation and methods of teaching. For purposes of 
the, analysis , the modules were divided irito four groups as showp in Table. 
II. Module groups 1 and 2 were taught in the fall of 1979; all module 
groups were taught during the winter and spring of 1980, although groups 3 
and 4. were, emphasized. Typically, students studied three to five modules 
eac'h. Treatment students were tested both before and ,after they stddied ' 
the particular modules taught in their classes. The test results of only 
those students who studied all modules in -a group and took both a pretest 
and. a posttest were used in the analysis.' 



TABLE II 
Module Groups 







Number of 
Modules 


*ZP 

u Topics 


Group 


1 


5 


history and philosophy' of vocational education 


Group 


2 


5* 


vocational education curriculum development " 


Group 


3 


2 


procedures for individualizing curricula for special students 


Group 


* i 


5 


evaluation and administration of vocational education 



*After the field test, modules in Group 3 were divided between Groups 1 aa<T2 resulting in 
the three series of modules described earlier. One module in Group 2 was deleted and its 
significant content, was integrated into several of the other modules, - resulting in the final 
total of 16 modules. 

J 
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Field Test Sites and r Participants 

The sponsors of the 15 VECS ii^ld test sites included twelve colleges 
or universities and two state departments of education. Five of the educa- 
tional institutions held classes on campus, six used off-campus locations 
such as area vocational schools, and;one university sponsored two sites, 
one on campus and one at a military base. Field test instructors were most' 
often faculty, tfut also included a director of a regional occupational pro-t 
gram, a principal of an area vocational school, and $ director of a state 
curriculum and instructional materials center. 

Students who participated iji the field test included: (1) undergradu- 
ates preparing to be vocational jpjfcucat ion teachers; (2) practicing voca- s 
tional education teachers, admjjjfcfetrators , counselors and curriculum devel- 
opers; and^(3j persons with occupational skills developed in business or 
the military who wished to teach their specialty. Students 1 goals inciaded 
obtaining a state vocational teaching certificate, a college degree at the 
undergraduate or graduate level, and inservice professional development. 
Weekly classes, held during t(ne day, at night, or on weekends, were the 
main instructional delivery method, although several students took the 
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TABLE III 
Field Toat Sltoa and Participants 



I 

I 



Sponsor 



1. University of Idaho 

2. Washington State Unlveralty 

3. California State University, 
S.ieraotento 

4. Virginia Polytechnic Institute . 
and State University 

5. Southern Illinois University, 
Carbondaie ' 

6« Oklahoma State Dope rt merit of Voca- 
tional and Technical Education 

7. Ok la horn a State. Unlveralty 

8. Southern Illinois Unlveralty, 
Carbondaie at Travis Air Force 

9. Unlveralty of South Florida 

10. Now Jersey Dupa rt taunt of 
Edit cat Ion 

11. SUNY Co li ego of Technology, 
Utica/koMo 

12. Unlveralty ot North Carolina, 
•Greensboro 

13* Ni'rth Carolina State University 

14, Texas' A & M Untvereity 

15. Enat TexAa State Unlveralty 



1-ocat ion; 
on or 
off campus ' 


Purpose of 
Inati tutiorr; 
Preaervice N (P) 
orTiiaei'vlceO) 


Student 


Number of Students Who ' 
Contributed Data to the 
Analysis, by Module Group 


Character 1st ics 


Group I 


Group 2 


Group 3 


Group A 


' off 


I 


\ ^ 

.Vocational teachers " ; 


5 . 


4 


7 


7 


* on 


" P 


* 

future vocational teachers 










off 


P 


Occupational specialists 




8 


1 1 




Off 


» f i 


Secondary and poatsecondary teachers 


10 


8 




• 


Jn 


p 


Occupational specialists and future 
business mJuctvt ion teachers 

1 ) 


— 


25 





9 


N/A 


i 


Staff AfT curriculum and instructional 
naterlala center . 


7 


6 


id 


- IB 


on 

» off 


i 


Vocational teachers ■ ^ 


4 


5 


10 


10 


p 


Mil 1 tar y occupational apeciullsta 
4 


23 


— 


— 




otf 


• i 


Vocational teachers, supervisors, 

and administrators 

• * 


— 


* — 


— 




* N/A 


i 


Vocational teachers, coordinators, 
and guidance counselors 


i 









on 


P & i 


Craduato students with vocational- 
^eclmlcaT backgrounds 


2 








Oil 


i 


llomu economics tenchcra «. 


5 


13- 






on 


i 


Teachora and state department admin- 
istrators 


- it 


2 






* . off 


♦ I 


Vocational toachers, admiuiat raters, 
and counaelora^^^y 


16 


10 


7 4 


8 


off 


I 


Vocational toachers and administra- 
tors v_ 

/ 


26 


* 12 


17 


32 


s 






109 


93 


• 70 


r 84 
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modules through independent study arrangements/ Class sizes ranged from 
over 30 to under *10. Classes varied with respect to; the amount of the 
teacher's participation inr instruction with the VECS nodules, the emphasis 
placed on module topics and activities, and the amount of supplementary 
marerial used. Table III summarizes information on the field test sites . 
and participants who supplied the data reported here*(Claudy, Blank, & 
Hamilton, in press-a; Claudy , Hamilton, Kelly, Cummings, & Cappello, in 
press-b). Data from all sites and all participants are not reported due to 
the decision to base the analysis on result's from only those students who 
studied all modules in a group* 
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Measurement of Effects 

Specially constructed instruments for use in the field test w^re 
developed to estimate the effectiveness of the VECS modules. Field test 
evaluation forms (FTEFs) measured the cognitive and affective outcomes of 
module study and collected biographical information on students. . The FTEFs 
provided information on the overall effect of studying groups of modules 
rather than an assessment of specific objectives. Effects of module groups 
-Jl and 2 were assessed by one set of FTEFs, while module groups 3 and 4 were 
tested by another set. * 

Content of the FTEFs ^ 

The FTEFs had six sections, each designed to collect a different kind 
of information. One section requested background information on eflucatioi), 
current occupation, and previous work experience. Another section provided 
a checklist ort which respondents indicated recent professiortal development 
experiences. Two sections requested respondents to rate themselves on 
twelve activities that are typical of the curriculum specialist's role. 
The first self-rating was of a person's competence ^in performing the 
activities; the second was of a person's desire to Engage iiT the activi-* 
ties. Competence self-ratings ranged from 1 to 6, based on agreement with 
•statements such as "I cannot perform this activity even with supervision or 
guidance 11 (1) to "I consider myself an expert in this activity and can 
teach it to others" 16). Attitude self-ratings ^ranged from 1 ("I would t 
like very much npt to do this") to 4 ("I wopld like very much to do this")* 

The remaining two sections contained multiple-choice and short-answer 
items based on significant content of the modules. In order to reduce 
response burden and increase the number of people who might respond, the 
* number of itejns included on any one FTEF was severely restricted. Only 
one, four-alternative, multiple-choice question was. included for each goal , 
of each module included in the group of modules tested by a form. The set 
of FTEFs testing module groups 1 and 2 contained 37 multiple-choice 
items-; FTEFs for module groups 3 and 4 contained 23 multiple-choice items. 
Two sets of parallel multiple-choice items were written for each 'module 
group. Treatment students took v one set of items as a pretest and the other 
set as a posttest. In order to control for possible differences in the 
difficulty of the two parallel sej:s, one set" was used as a pretest at about 
half the sites, while the other set was included in the pretest at the 
remaining sites. f 

o '* '' . 11 " 8 - ' . 



^ One short-answer item was written for each of the modules in grpups 
2-4. ✓As these items were intended to assess cognitive prerequisites of the 
performance of skills central to the curriculum specialist's role, no shortr 
answer items were included to test outcomes of the group 1 modulesfcbecause 
these modules provide background information and do not focus oh skills, 
The short-answer skill items required students. to list such things as the 
appropriate procedures for certain situations, the sequential s£eps in a 
protedure, or the advantages and disadvantages of several alternative pro- 
cedures. Responses were rated on a scale of 1-5 based on pre,specif ied 'cri- 
teria that reflected the quality of the response in terms of its accuracy 
and completeness. 

. • A . 

Development of FJEFs 

"£he FTEFs were prepare^ according to a careful, sxep-by-step develop- 
ment process, and were approved by the^ Federal Education Data Acquisition 
Council (FEDAC) , the governmental agency charged with ensuring that data 
are collected* by the most efficient and effective means. The statements of r 
a curriculum specialises activities used in the self-assessments of compe- 
tence and attitude^were^derived from the content of the modules, which was 
in turn derived from the detailed competency analysis performed prior to 
the writing of the initial sfet of modules. The multiple-choice and perfor- 
mance items were written based on objectives and topics 'the modules 1 authors 
considered most significant. Each item 4 was reviewed by the module's author 
for content accuracy and by the project's evaluation director for technical 
adequacy. Items were then revised as many times as was necessary.^ 

The multiple-choice items were pilot-tested. Respondents answered, 
critiqjied, and edited the* items. Point-biserial correlations and diffi- 
culty levels were calculated for each item; items with low correlations or 
very high or low difficulty levels were removed or improved. An attempt 
was made to balance difficulty levels of multiple-choice item sets when 
constructing the alternative forms of the FTEFs. However, 4 we could not 
obtain clear evidence that parallel multiple-choice item sets were of equal 
difficulty. Therefore, cases were dropped randomly from the analysis so 
that equal numbers of students took each set of items. 

* Spearraan-Brown estimates of the split-half reliabilities of the two 
parallel multiple-choice item>sets were calculated for each of the four 
module groups. The resulting eight correlations ranged from .12 to .55 
with 'approximately .40 the mode and .33 the mean. Two decisions macie in 
designing the F^EFs operated against obtaining high split-half reliability 
estimates. First, the number of multiple-choice items included' on any* one 
form was deliberately kept low. The numbers of items for module groups 1t4 
(see Table II) were 17 * 20, 7 and 16 respectively. The Spearman-Brown 
reliability estimate for a sixty-item' test with the same split-half corre- 
lation as the 17 items written for module group r would be effectively ^ 
double The actual . estimate obtained. The second decision operating against 
obtaining high split-half reliability estimates concerned using ^ne set of 
items to test the content of a number of modules. The items testing a 
group of modules did not include interchangeable measures $>f the same 
learning, and since the Spearman-Brown statistic essentially is a measure 



of internal consistency, a high split-half< reliability estimate could not 
be" expected. * ^ 

Despite the low reliabilities calculated for. the multiple-choice items, 
two points should be kept in mind. First, .the content of the test items 
was judged valid by the module authors and the project director, those per- 
sons most familiar with the concepts the modules were designed to t;each. 
Second, the FTEFs were the only instrument^ available to assess the effects 
of the modules. No standardized tests existed that measured the goals of 
these module? in a systematic manner. 



Effects of Module Study 

» 

The results of the field £est show that study of the VECS modules pro- 
duce^ significant gains in knowledge of the history and philosophy of voca- 
tional education and topics in vocational education curriculum development, 
management, and evaluation. Use of the modules also increases students 1 
self-perception of personal competence in activities performed by a curri- 
culum specialist. The main effects of the VECS modules are cognitive and 
are seen in the results obtained oq the multiple-choice and shorf-answejjjf 
items. The cognitive effects are reinforced by an affective outcome 
revealed in the se lf-ratings £t competence. 

The assertion of the effectiveness of the VECS modules is based on the 
comparison of the pretest and posttes't results of students who studied the 
modules. Treatment group pretept/posttest comparisons are appropriate for 
showing that newly .developed <instructidkal materials do, in fact, produce 
the kind of results; for which they were designed^ For the VECS field test, 
control group data were not intended to demonstrate that the modules are 
more effective than a competing treatment, buAr^ther to indicate that 
factors other than module study occurring during the*coursie of the'field 
test were unlikely to have produced the obtained results. Fortunately, 
factors in the design of the field test that are discussed later indicate 
it is likely that ^the field test results were indeed, produced by module 
study. * , 



Data Analysis 

■ 1 ) 

^0$fcjK.n preparation for the analysis,' several summary ^scores were created. 
* 'Module group scores on multiple-choice items were calculated for treatment 
students who studied all modules in a group. Similar scores were created 
fot short-answer items. Summary scores were created for the self-ratings 
of competence and attitude by adding" together the ratings made on each ol: 
the 12 activity statements. The total number of professional development 
experiences reported was also summarized in a score. m 

,An analysis for each module group was conducted. Treatment group 
students 1 pretest results were compared with their posttest results using 
• t-tests for paired s«»Rles. Treatment pre versus treatment post compari- 
sons were analyzed for multiple-choice and short^answer itein module group 
scores, and competence and attitude self-rating summary score^r. 

O '' ' ' -10- "« 



Results 



Data from the treatment groups pretest and posttest comparisons are 
shown in Table IV. Multiple-thoice item comparisons for all butt one of the 
•module groups show gaips significant at the .05 level or better. The 



TABLE IV 



Results of 


Treatment Group Pretest 


vs. Treatment Grou* Posttest Paired Cocoarisens 
















** 




Number of 
s Subjects 


Highest 
Possible 

Score 




Mean 


Standard 
Deviation 


3. result 3 on '-tilt-sie-Cioics I tans 


Module Group 1 
c • 4.28 d 


\ 

Pretest 
Postt&e^ 


96 
96 , 


17 
17 




7.08 
8.29 


2.30 
2.44 


Module Group 2 - 
c - 3.75c 


Pretest 
Posttest 


78 


zu 
20 




7 7 A 
9.0.9 


2*67 
2.65 


t - U92 , 


Pre tes t 
Posttest 


69 

69 , 


7 
7 




5.io 

3.65 


1.30 
1.84 


t - 2.23 a 


Pre tes t 
Posttest 


68 
68 


16 




7.78 
'8.63 


2.33 
3.19 


b. Results on Shorz 


-Ansve? liens 












Module Group 2 
c ■ 6.90 d 


Pretest 
Posttest 


77 
77 


25 
25 




5.12 
S\ 64 


\ 

3.50 
4.68 


'Module Group 3 1 
t • 4.97 d J 


Pretest 
Posttest 


69 
69 


10 
10 


\ 


2.97 
5.33 


2.15 
3.37 


Module Gcoup 4 
c • 6.19 a 


Pretest 
Posttest 


68 
68 


25 
25 




J .00 
8.74 


5.68 


e. Results on Self- 


Ratince of Convetenoe 












Module Group L 
t - 6.14 d * 


Pretest ^ 
Posttest 


89 
89 


72 
72 




42.13 
47.93 • 


11.72 
9.83 


Module Group 2 
-t - 8.3i d 


Pretest 

Posttest j 


69 " 
69 


72 
72 




39.62 
49.46 


11.06 
t 8#34 


Module Group 3 
t - 2.20* 


Pretest 
Posttest 


65 
65 


72 \ 




43.35 
46.52 


J 10*84 
10.16 


*t 

Module Group 4 
c • 2.90° 


Pretest % 
Posttest 


60 
60 


72 
72 




42.'o7 
45.60 


11.44 
11.56 


£ 'Results on Self- 


•Patincs of Attitude 




S 








Modul% Group 1 
t • -0.41 


Pretest 
Posttest 


80 
80 


48 
48 




36.93 
36.69 


4.26 
4.84 


Module Group 2 
c • -0.81 


Pretest 
Posttest 


57 
57 


48 
48 




38.14 
37.39 


5.94 
* 6.23 


Module Group 3 
c • 2.01* 


Pretest 
Edsttest 


49 
49 


48 I 
48 




34.53 
36.06 


6*03 
4.64 


Module Group 4 
c • 0.62 


Pretest 

Posttest 

t 


54 
54 


48 
48 




33.2a 
33.69 " 


6.28 
7.4<T 


^Significant at the 
Significant ac the 


.05 level 
.01 level 




^Significant at the 
Significant at the 


.001 
.0001 


level 
level 





r 



-1* 

U 



remaining comparison approached significance (p s .058). Since this group is 
cqmpose'd of only two modules, the small, number of items (7) probably 
acc/ounted for the lack of a significant gain. Comparisons of results on 
dhort-answer itetps for all module groups were significant at better than 
tne .0001 lev^l^y Comparisons of competence self-ratings for all module 
groups showed significant gains, but those for attitude were inconclusive. 

^ > 

Disc ission of Field Tegt Results 

P % 

Table V presents information on treatment group students who provided 
data for the pretest/posttest comparisons upon which the assertlfc^ of the 
effectiveness of the VECS modules is based. These individuals represent 
the broad range of educational attainment and experience in vocational edu- 
cation Chat would be possessed by the population of vocational etfufcators 
and potential vocational educators for whom the modules are intended to 
provide training. Because the modules were tested on such a heterogeneous 
group of individuals, the /esults of the field test should be generalizable 
to the entire target population. The assertion of the statistical reli- 
ability of the field test results is based on the fact that the field test 
was conducted under natural conditions representing the wide variety of 
conditions foj: which the modules were*designed. Each of the classes in 
which the modules were used can be considered a replication of the field 
test. Data from each of the replications were combined rather -than anal- 
yzed separately* due to the small number of subjects withboth pretest and 
posttest data at each site. Nevertheless, because the modules were tested 
at different times and different places, it is likely that the results 
reported are not limited to the field test. 

y 

The strongest evidence that the \ield test results were ,due to module 
study comes from factors in^the design of the field test. Since the field 
test was conducted at different times at a number of sites, the effects of 
events other than module study would tend to average out. Practice effects 
on multiple-choice items are unlikely since different items comprised the 
pretest and posttest instruments. Practice effects on short-answer items 
and maturation effects are unlikely to have caused field test results 
because the tests are so closely tied to the cfcnte^nt of the modules that 
gains resulting from other sources are improbable. 



Conclusions 

The results of the national field test demonstrated that the VECS mod- 
ules increase knowledge of topics necessary to the successful performance 
of skills central to the curriculum specialist role. They also tended to 
increase peoples 1 confidence in their ability to perform these skills. As 
vocational education expands and changes, more curriculum specialists will 
be needed to create and maintain high quality curricula. Curriculum spe- 
cialists will receive training under many different circumstances. The 
VECS modules provide a low cost,^ soundly developed resource that is 
adaptable to a variety of needs. The modules summarize and organize a 
great body of material permitting vocational education curriculum special- 
ist training to be conducted systematically at many different types of 
locations* 



TABLED 

Characterlatica of Students in the Treatment Group 







Sex, 


Degree 


.Credential 


Previous Experience 
in Vocational Ed 


Current Position 






Ho. 


• 

Z 

K 


• 

.} 


<Z ** 
Assoc. 


z 

finch. 


Z* 
Mast« 


Z 

Dr. 


Z 

Teach- 
ing 


• Z 
Adminis- 
trative 


X 

X Aduinis- 
Teaclter trator 


Z 

Curric- 
ulum 

Devel- 
oper ' 


Z 

Teacher 


Z 

Adminis- 
trator 


* 

Curric- 
ulum 

Devel- 
oper 


Z 

Under- 
grade - 
Student 


X 

Cnade 
Student 




Module 
Croup 1 


96 


44 


56 


5 




27 


1 


67 


7 


59 9 


9 


57 


14 


9 


21 


46 




Module. 
Croup 2 


78 


47 


53 


23 




26 


A 1 


74 


5 








9 


15 


26 


44*- 




Module 
Croup 3 ' 


69 


46 


54 


13 

*> 


45 


*rs 


1 


78 


13 


67 14 


23 


62 


' 14 


19 


6 


41 




Module 
Croup 4 


68 

i 


41 


59 


6 , 


44' 


,44 


4 . 


75 


9 


46 12 


IB 


51 


15 


15 


\ 


54 





JWTK: Percentages do roe always add to 100Z due to rounding, «isaing data, or marking -ore than one category for Degree, Credential, 
Previous Experience, -or Current Poaition. . B ' 

- ■ / 
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Dissemination of Materials * 

\^ 

To introduce vocational educators to the VECS materials, AIR staff - 
conducted a series of technical- assistance forums and dissemination work- 
shops allocations across the country. At the six one-day forums, field 
test instructors and site coordinators gave detailed accounts of how to 
use the VECS modules in various settings. Approximately 250 vocatiorial 
educators from 53 states and territories attended the workshops, including 
state-level vocational education administrators, state liaison represen- 
tatives of the NNCCVTE, college and university faculty, and' administrators 
from large school districts. The participants left the workshops with 
plans for using and disseminating the materials throughout their states. 

An agreement with the East Central Network Curriculum Centfer, a 
member of the NNCCVTE, to print and distribute the VECS materials" on a/ 
cost-recovery basis was approved by the Copyright Authorization Office, 
U.S. Department of Education. ' 



******.*** 

7 \ * • * 

r \ ' 

7 

' - i 

' . Footnotes 

1. The information reported here was obtained pursuant to Contract No\ 300- 
78-0562 with the Office" of Education (now Department of Education), U.S. 
Department of Health, Education, and Welfare. Information, points of 
view, or opinions stated do not necessarily represent official Depart- 
ment of Education position or polriqy;* 

.5 * < 

,2. The 16 VECS modules and the user's guide are available for $35.00 from 
the: 

East Central Network Curriculum Center f " , * 

Sangamon State University J E-22 
Springfield, IL c62708 

Single or multiple copies of individual modules )ar series of modules 
are also available. The costs of -reference materials listed in the 16 
modules would total about $125. 
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